feat(trtllm): run TRT-LLM's mooncake-store connector on the native mooncake-master service - #595

Problem

SGLang and vLLM recipes share a Mooncake Store pool by declaring a mooncake-master service. TRT-LLM recipes cannot: on main, a TRT-LLM recipe with that service fails to load with engine.mooncake_kv_store: Unknown field. This PR connects that managed service to TRT-LLM by supplying the final Mooncake client config to every worker rank, for both trtllm-serve and dynamo.trtllm.

TRT-LLM already supports joining a separately launched native mooncake_master through two forms of kv_connector_config.mooncake_store.pool:

• A bare host:port, which uses the default rdma protocol and P2PHANDSHAKE metadata server.
• file:///path/to/pool.json, where the pool manifest supplies the master address and optional connection settings.

Neither form requires launching the master through trtllm-serve mooncake_master (pool resolution and manifest schema). For either form, TRT-LLM combines the pool settings with role, model_key, segment_size, and other settings from the engine's mooncake_store block to render the final client JSON (renderer). That provisioning step is invoked by trtllm-serve; ordinary dynamo.trtllm workers do not run it.

This PR uses the existing direct-client-config path: srtctl renders the final JSON from the service and per-role settings and exports MOONCAKE_CONFIG_PATH on each worker's srun task, so all launcher-created ranks inherit it. This JSON is the connector's client config, not the pool.json manifest. The connector reads the supplied file as authoritative (config.py); trtllm-serve skips its provisioning when the variable is set and ignores the engine's mooncake_store block (master.py). Using this path extends srtctl's existing vLLM config-delivery mechanism to TRT-LLM and supports both frontends without adding serve-only provisioning or per-server run_dir management.

What a TRT-LLM recipe looks like with this PR

From examples/trtllm/trtllm-serve-disagg-mooncake.yaml (Mooncake-relevant keys only):

engine:
  type: trtllm
  served_model_name: "Qwen/Qwen3-0.6B"     # also the pool's model_key (default)
roles:
  prefill:
    args:                                  # TRT-LLM's own config: switch the connector on
      kv_cache_config:
        use_kv_cache_manager_v2: true
      cache_transceiver_config:
        backend: "NIXL"
        transceiver_runtime: "PYTHON"
      kv_connector_config:
        connector: mooncake-store          # no mooncake_store block
  decode:
    args:                                  # the same three keys
      kv_cache_config:
        use_kv_cache_manager_v2: true
      cache_transceiver_config:
        backend: "NIXL"
        transceiver_runtime: "PYTHON"
      kv_connector_config:
        connector: mooncake-store
    mooncake_store_config:                 # new: this role's client keys
      global_segment_size: 16GiB           # decode ranks lend more host memory
services:
  - name: mooncake-master                  # the same service SGLang and vLLM use
    type: mooncake-master
    options:
      store_config:                        # client keys shared by every role
        protocol: rdma
        global_segment_size: 8GiB          # per rank; byte counts or binary units
        stage_through_host: true

Before workers start, srtctl writes one client config per role that selects the connector (output of the mock orchestrator for this example):

/logs/mooncake_store_config_prefill.json       /logs/mooncake_store_config_decode.json
{ "role": "both",                              { "role": "capacity",
  "model_key": "Qwen/Qwen3-0.6B",                "model_key": "Qwen/Qwen3-0.6B",
  "protocol": "rdma",                            "protocol": "rdma",
  "global_segment_size": "8GiB",                 "global_segment_size": "16GiB",
  "stage_through_host": true,                    "stage_through_host": true,
  "master_server_address": "<infra>:8700" }      "master_server_address": "<infra>:8700" }

Each file layers, later wins: the role default (both for prefill and agg, capacity for decode) and model_key (the served model name), the service's options.store_config, then roles.<role>.mooncake_store_config; master_server_address is always the master srtctl launched. The master is the usual mooncake_master --port=8700 --enable_http_metadata_server=true ... on the infra node, gated on ports 8700, 8701, and 8702. Each worker's srun task gets its role's file, so every rank trtllm-llmapi-launch starts inherits it:

export MOONCAKE_CONFIG_PATH=/logs/mooncake_store_config_prefill.json && ...
exec trtllm-llmapi-launch trtllm-serve <model> --host 0.0.0.0 --port 6100 --tensor_parallel_size 1 \
  --config /logs/trtllm_config_prefill.yaml --served_model_name Qwen/Qwen3-0.6B

With frontend.type: dynamo the same recipe launches trtllm-llmapi-launch python3 -m dynamo.trtllm ... --extra-engine-args /logs/trtllm_config_prefill.yaml with the same file and variable (examples/trtllm/dynamo-disagg-mooncake.yaml); no wrapper is needed because dynamo.trtllm provisions nothing and the connector reads the variable.

How the structure lines up with SGLang and vLLM

Per-role settings follow the rest of the schema: they live under roles.<role>, layered over a shared default by one accessor (role_mooncake_store_config, like role_kv_events). Keys every server of the pool must agree on (model_key, namespace, metadata_server, protocol, master_server_address) are refused there and stay in options.store_config.

Changes

• TRTLLMBackend.mooncake_kv_store (TRTLLMMooncakeKVStoreConfig), filled from a declared mooncake-master service.
• RoleConfig.mooncake_store_config (TRT-LLM only), read through role_mooncake_store_config; refused on SGLang and vLLM roles with a pointer to their existing knobs.
• Backend.mooncake_store_configs(infra_node_ip, served_model_name) lists the files the orchestrator writes; get_mooncake_worker_env takes the worker mode so TRT-LLM can point each role at its own file.
• model_key defaults to the served model name. TRT-LLM deliberately has no default (config.py) so two checkpoints sharing a pool cannot read each other's KV; this pool belongs to one job, and an external master is refused for TRT-LLM.
• Load-time checks for what TRT-LLM would otherwise reject in every rank after the model loads: a mooncake_store block or its block-only keys (segment_size, pool, run_dir, master_timeout) next to the master, an empty model_key, an unknown role, sizes that are not byte counts or binary units (TRT-LLM reads GB as 1000³ and refuses it), non-positive transfer_batch_size or local_buffer_size, and stage_through_host given as a string. Without a master service, a connector role must find a client config itself (MOONCAKE_CONFIG_PATH in its env, or a mooncake_store block with its own run_dir when trtllm-serve runs the engine).
• Two examples (trtllm-serve, Dynamo), docs (docs/mooncake-kv-store.md, docs/services.md), regenerated schema docs, tests, and launch snapshots.

Validation

• Tests: 4,485 passed, 2 skipped; ruff check, ruff format --check, and the schema-docs check pass.
• ty check src/srtctl/ passes with ty 0.0.86, the release CI installs. The new RoleSettings.mooncake_store_config is declared as a read-only property for the type checker only, like the other RoleSettings attributes since fix(types): declare RoleSettings and Backend.roles read-only for ty 0.0.86 #597.
• make snapshots: snapshots for the two new examples are included; the per-role field and model_key default change no srun launch of any example.
• On GPUs, both frontends (trtllm-serve and Dynamo), Qwen3-0.6B with one prefill and one decode worker on a GB300 node, TRT-LLM 1.3.0rc23.post1 built from feat/m3_with_msa at 1ab6bbc990 (the connector API from [None][feat] BREAKING: Refine Mooncake connector API TensorRT-LLM#19706) and Mooncake 0.3.13: the native master took 1,375 pages (5.05 GB across the prefill segment and the decode capacity segment); after the prefill GPU cache evicted them, repeated requests read 1,374 pages (5.04 GB) back from the pool with identical greedy outputs. Both client configs that run wrote (prefill and decode) are identical to what this branch renders for the same recipe, with the run's decode segment size written as roles.decode.mooncake_store_config, apart from the master's IP address.

Limitations

• roles.<role>.mooncake_store_config is read by TRT-LLM only in this PR. vLLM still writes one file for every role, and SGLang takes Mooncake settings as environment variables.
• srtctl still exports MOONCAKE_MASTER, MOONCAKE_TE_META_DATA_SERVER, and MOONCAKE_LOCAL_HOSTNAME to TRT-LLM workers; TRT-LLM reads only the file, so they are inert there (documented).
• Engine-side limits are TRT-LLM's and differ by build, so srtctl does not repeat them. TRT-LLM main ([TRTLLM-12891][feat] Integrate Mooncake connector into KVCMv2 TensorRT-LLM#19171) keeps the same file contract; GPU validation used the feat/m3_with_msa build.